Methods and devices for data compression using context-based coding order

ABSTRACT

A method is presented for entropy coding data using an entropy coder to encode an input sequence. A context model is used to determine the context of each symbol and a probability estimation is made for each symbol. A method is presented for revising the coding order to be context-based, grouping symbols consecutively on the basis that they have a common context. A method is presented for entropy decoding a bitstream of encoded data encoded using a context-based coding order.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority to U.S. provisional patentapplication Ser. No. 61/363,717, owned in common herewith and thecontents of which are hereby incorporated by reference.

FIELD

The present application generally relates to data compression and, inparticular, to an encoder, a decoder and methods of coding and decodingusing a context-based coding order.

BACKGROUND

Data compression, whether lossy or lossless, often uses entropy codingto encode a decorrelated signal as a sequence of bits, i.e. a bitstream.Efficient data compression has a wide range of applications, such asimage, audio, and video encoding. The current state-of-the-art for videoencoding is the ITU-T H.264/MPEG AVC video coding standard. It defines anumber of different profiles for different applications, including theMain profile, Baseline profile and others. A next-generation videoencoding standard is currently under development through a jointinitiative of MPEG-ITU: High Efficiency Video Coding (HEVC).

There are a number of standards for encoding/decoding images and videos,including H.264, that employ lossy compression processes to producebinary data. For example, H.264 includes a prediction operation toobtain residual data, followed by a DCT transform and quantization ofthe DCT coefficients. The resulting data, including quantizedcoefficients, motion vectors, coding mode, and other related data, isthen entropy coded to generate a bitstream of data for transmission orstorage on a computer-readable medium. It is expected that HEVC willalso have these features.

A number of coding schemes have been developed to encode binary data.For example, JPEG images may be encoded using Huffman codes. The H.264standard allows for two possible entropy coding processes: ContextAdaptive Variable Length Coding (CAVLC) or Context Adaptive BinaryArithmetic Coding (CABAC). CABAC results in greater compression thanCAVLC, but CABAC is more computationally demanding. In any of thesecases, the coding scheme operates upon the binary data to produce aserial bitstream of encoded data. At the decoder, the decoding schemereceives the bitstream and entropy decodes the serial bitstream toreconstruct the binary data.

It would be advantageous to provide for an improved encoder, decoder andmethod of entropy coding and decoding.

BRIEF DESCRIPTION OF THE DRAWINGS

Reference will now be made, by way of example, to the accompanyingdrawings which show example embodiments of the present application, andin which:

FIG. 1 shows, in block diagram form, an encoder for encoding video;

FIG. 2 shows, in block diagram form, a decoder for decoding video;

FIG. 3 shows a block diagram of an encoding process;

FIG. 4 shows a simplified block diagram of an example embodiment of anencoder;

FIG. 5 shows a simplified block diagram of an example embodiment of adecoder;

FIG. 6 illustrates a zig-zag coding order for a 4×4 block ofcoefficients;

FIG. 7 shows, in flowchart form, an example method of entropy encodingan input sequence of bits;

FIG. 8 shows, in flowchart form, an example method of entropy decoding abitstream of encoded data;

FIG. 9 shows, in flowchart form, an example method of entropy encodingsignificance maps for a group of blocks of quantized DCT coefficients ina video encoding process; and

FIG. 10 shows, in flowchart form, an example method of entropy decodinga bitstream of encoded video data to reconstruct significance maps for agroup of blocks of quantized DCT coefficients in a video decodingprocess.

Similar reference numerals may have been used in different figures todenote similar components.

DESCRIPTION OF EXAMPLE EMBODIMENTS

The present application describes devices, methods and processes forencoding and decoding binary data.

In one aspect, the present application describes methods of encoding aninput sequence of symbols using a context-based coding order.

In a further aspect, the present application describes a method forencoding an input sequence of symbols from a finite alphabet using acontext model. The method includes determining a context for each symbolof the input sequence as specified by the context model, wherein thecontext model specifies the context based upon information other thanother symbols in the input sequence; and for each context, encodingconsecutively those symbols of the input sequence having that context,to produce encoded data, wherein, for each symbol, encoding includesdetermining the probability of that symbol and encoding that symbolbased on its determined probability, and wherein at least some of thesymbols from the input sequence have a same context.

In yet another aspect, the present application describes a method ofentropy encoding an input sequence of symbols using a context model,wherein the context model specifies a context for each symbol. Themethod includes reordering the input sequence to group the symbols onthe basis of context to form groups of symbols, each group beingassociated with a respective one of the contexts; for each of the groupsof symbols, determining a probability associated with each symbol in thegroup of symbols; and, for each of the groups of symbols, encodingconsecutively the symbols in that group based upon their determinedprobabilities, to produce encoded data.

In another aspect, the present application describes an encoder forencoding an input sequence of symbols. The encoder includes a processor;memory; and an encoding application stored in memory and containinginstructions for configuring the processor to encode the input sequencein accordance with one of the methods described herein.

In yet another aspect, the present application describes methods ofdecoding a bitstream of encoded data generated using a context-basedcoding order.

In one aspect, the present application describes a method for decodingencoded data to reconstruct a sequence of symbols, wherein the encodeddata has been encoded in accordance with a context model, wherein thecontext model specifies the context for each symbol. The methodincludes, for each of two or more of the contexts defined in the contextmodel, decoding a portion of the encoded data to obtain a group ofconsecutive symbols having that context, wherein, for each of theconsecutive symbols, decoding includes determining a probabilityassociated with that symbol and decoding that symbol from the encodeddata based on that determined probability; and reconstructing thesequence of symbols by reordering the symbols of the groups ofconsecutive symbols.

In another aspect, the present application discloses a method for methodfor decoding a bitstream of encoded data to reconstruct a sequence ofbits, wherein the encoded data has been encoded in accordance with acontext model and a coding order. The method includes decoding theencoded data to obtain subsequences of decoded bits, wherein eachsubsequence of decoded bits is associated with a probability;reconstructing a group of consecutively encoded bits from thesubsequences of decoded bits based upon the group of consecutivelyencoded bits having a same context and by determining a probability foreach bit in the group; and rearranging bits of the reconstructed groupon the basis of the coding order to produce a decoded bitstream.

In a further aspect, the present application describes a method fordecoding a bitstream of encoded data to reconstruct a sequence ofsymbols, wherein the encoded data has been encoded in accordance with acontext model. The method includes determining a context for each symbolto be reconstructed as specified by the context model, wherein thecontext model specifies the context for each symbol; for each determinedcontext, determining the respective estimated probability of two or moresymbols having that context, and obtaining each of the two or moresymbols by decoding the encoded data in accordance with the estimatedprobability of that symbol; and reconstructing a sequence of decodedsymbols from the symbols obtained.

In yet another aspect, the present application describes a method forentropy decoding a bitstream of encoded video data to reconstruct a bitsequence of video data, wherein the encoding is based on a contextmodel, wherein the encoded video data comprises codewords, wherein thebitstream of video data includes significance maps for a group ofcoefficient blocks, and wherein each significance map includes a sigvector and a last vector. The method includes decoding the codewords toobtain subsequences of decoded bits, wherein each subsequence of decodedbits is associated with a probability; and reconstructing thesignificance maps in a context-based order by for each group of bits inthe significance maps having a same context, determining a probabilityfor each bit in that group, and forming each of the groups of bits frombits selected from the subsequences of decoded bits based on thedetermined probabilities.

In yet a further aspect, the present application describes a decoder fordecoding encoded data for decoding a bitstream of encoded data toreconstruct a sequence of symbols. The decoder includes a processor;memory; and a decoding application stored in memory and containinginstructions for configuring the processor to decode the bitstream inaccordance with one of the methods described herein.

In yet a further aspect, the present application describescomputer-readable media storing computer-executable program instructionswhich, when executed, configured a processor to perform the describedmethods of encoding and/or decoding.

Other aspects and features of the present application will be understoodby those of ordinary skill in the art from a review of the followingdescription of examples in conjunction with the accompanying figures.

The following description relates to data compression in general and, inparticular, to the efficient encoding and decoding of finite alphabetsources, such as a binary source. In many of the examples given below,particular applications of such an encoding and decoding scheme aregiven. For example, many of the illustrations below make reference tovideo coding. It will be appreciated that the present application is notnecessarily limited to video coding or image coding. It may beapplicable to the any type of data subject to a context-based encodingscheme. In particular, it may be applicable to any context-basedencoding scheme in which the context model provides that the context ofsymbols in a sequence X depends upon side information. Side informationmay include, for example, coefficient position in a matrix, a bithistory for coefficients in that position, etc.

The example embodiments described herein relate to data compression of afinite alphabet source. Accordingly, the description often makesreference to “symbols”, which are the elements of the alphabet. In somecases, the description herein refers to binary sources, and refers tothe symbols as bits. At times, the terms “symbol” and “bit” may be usedinterchangeably for a given example. It will be appreciated that abinary source is but one example of a finite alphabet source. Thepresent application is not limited to binary sources.

In the description that follows, example embodiments are described withreference to the H.264 standard. Those ordinarily skilled in the artwill understand that the present application is not limited to H.264 butmay be applicable to other video coding/decoding standards, includingpossible future standards, such as HEVC. It will also be appreciatedthat the present application is not necessarily limited to videocoding/decoding and may be applicable to coding/decoding of any finitealphabet sources.

In the description that follows, in the context of video applicationsthe terms frame and slice and picture are used somewhat interchangeably.Those of skill in the art will appreciate that, in the case of the H.264standard, a frame may contain one or more slices. It will also beunderstood that under other video coding standards, such as HEVC,different terms such as picture may be used instead of frame or slice.It will further be appreciated that certain encoding/decoding operationsare performed on a frame-by-frame basis, on a slice-by-slice basis, oron a picture-by-picture basis depending on the particular requirementsof the applicable video coding standard. In any particular embodiment,the applicable video coding standard may determine whether theoperations described below are performed in connection with frames,pictures and/or slices, as the case may be. Accordingly, thoseordinarily skilled in the art will understand, in light of the presentdisclosure, whether particular operations or processes described hereinand particular references to frames, slices, or both are applicable toframes, slices, pictures or any of them for a given embodiment.

In many of the examples herein, the processing may be based upon acoding unit, transform block, tile, macroblock, or other grouping orsubgrouping of symbols or sequences appropriate for a givenimplementation.

Reference is now made to FIG. 1, which shows, in block diagram form, anencoder 10 for encoding video. Reference is also made to FIG. 2, whichshows a block diagram of a decoder 50 for decoding video. It will beappreciated that the encoder 10 and decoder 50 described herein may eachbe implemented on an application-specific or general purpose computingdevice, containing one or more processing elements and memory. Theoperations performed by the encoder 10 or decoder 50, as the case maybe, may be implemented by way of application-specific integratedcircuit, for example, or by way of stored program instructionsexecutable by a general purpose processor. The device may includeadditional software, including, for example, an operating system forcontrolling basic device functions. The range of devices and platformswithin which the encoder 10 or decoder 50 may be implemented will beappreciated by those ordinarily skilled in the art having regard to thefollowing description.

The encoder 10 receives a video source 12 and produces an encodedbitstream 14. The decoder 50 receives the encoded bitstream 14 andoutputs a decoded video frame 16. The encoder 10 and decoder 50 may beconfigured to operate in conformance with a number of video compressionstandards. For example, the encoder 10 and decoder 50 may be H.264/AVCof HEVC compliant. In other embodiments, the encoder 10 and decoder 50may conform to other video compression standards, including evolutionsof the H.264/AVC standard like HEVC.

The encoder 10 includes a spatial predictor 21, a coding mode selector20, transform processor 22, quantizer 24, and entropy coder 26. As willbe appreciated by those ordinarily skilled in the art, the coding modeselector 20 determines the appropriate coding mode for the video source,for example whether the subject picture/frame/slice is of I, P, or Btype, and whether particular macroblocks or coding units within theframe, slice, picture or tile are inter- or intra-coded. The transformprocessor 22 performs a transform upon the spatial domain data. Inparticular, the transform processor 22 applies a block-based transformto convert spatial domain data to spectral components. For example, inmany embodiments a discrete cosine transform (DCT) is used. Othertransforms, such as a discrete sine transform or others may be used insome instances. Applying the block-based transform to a block of pixeldata results in a set of transform domain coefficients. The set oftransform domain coefficients is quantized by the quantizer 24. Thequantized coefficients and associated information, such as motionvectors, quantization parameters, etc., are then encoded by the entropycoder 26.

Intra-coded frames/slices/pictures (i.e. type I) are encoded withoutreference to other frames/slices/pictures. In other words, they do notemploy temporal prediction. However intra-coded frames/pictures do relyupon spatial prediction within the frame/picture, as illustrated in FIG.1 by the spatial predictor 21. That is, when encoding a particularblock/coding unit the data in the block/coding unit may be compared tothe data of nearby pixels within blocks already encoded for thatframe/slice/picture. Using a prediction algorithm, the source data ofthe block may be converted to residual data. The transform processor 22then encodes the residual data. H.264, for example, prescribes ninespatial prediction modes for 4×4 transform blocks. In some embodiments,each of the nine modes may be used to independently process a block, andthen rate-distortion optimization is used to select the best mode.

The H.264 standard also prescribes the use of motionprediction/compensation to take advantage of temporal prediction.Accordingly, the encoder 10 has a feedback loop that includes ade-quantizer 28, inverse transform processor 30, and deblockingprocessor 32. These elements minor the decoding process implemented bythe decoder 50 to reproduce the frame/slice/picture. A frame store 34 isused to store the reproduced frames/pictures. In this manner, the motionprediction is based on what will be the reconstructed frames/pictures atthe decoder 50 and not on the original frames/pictures, which may differfrom the reconstructed frames/pictures due to the lossy compressioninvolved in encoding/decoding. A motion predictor 36 uses theframes/slices/pictures stored in the frame store 34 as sourceframes/slices/pictures for comparison to a current frame/picture for thepurpose of identifying similar blocks. Accordingly, for macroblocks orcoding units to which motion prediction is applied, the “source data”which the transform processor 22 encodes is the residual data that comesout of the motion prediction process. The residual data is pixel datathat represents the differences (if any) between the reference block orreference coding unit and the current block/unit. Information regardingthe reference frame and/or motion vector may not be processed by thetransform processor 22 and/or quantizer 24, but instead may be suppliedto the entropy coder 26 for encoding as part of the bitstream along withthe quantized coefficients.

Those ordinarily skilled in the art will appreciate the details andpossible variations for implementing H.264 encoders.

The decoder 50 includes an entropy decoder 52, dequantizer 54, inversetransform processor 56, spatial compensator 57, and deblocking processor60. A frame buffer 58 supplies reconstructed frames/pictures for use bya motion compensator 62 in applying motion compensation. The spatialcompensator 57 represents the operation of recovering the video data fora particular intra-coded block from a previously decoded block.

The bitstream 14 is received and decoded by the entropy decoder 52 torecover the quantized coefficients. Side information may also berecovered during the entropy decoding process, some of which may besupplied to the motion compensation loop for use in motion compensation,if applicable. For example, the entropy decoder 52 may recover motionvectors and/or reference frame information for inter-coded macroblocksor coding units.

The quantized coefficients are then dequantized by the dequantizer 54 toproduce the transform domain coefficients, which are then subjected toan inverse transform by the inverse transform processor 56 to recreatethe “video data”. It will be appreciated that, in some cases, such aswith an intra-coded macroblock or coding unit, the recreated “videodata” is the residual data for use in spatial compensation relative to apreviously decoded block within the frame/picture. The spatialcompensator 57 generates the video data from the residual data and pixeldata from a previously decoded block. In other cases, such asinter-coded macroblocks or coding units, the recreated “video data” fromthe inverse transform processor 56 is the residual data for use inmotion compensation relative to a reference block from a differentframe/picture. Both spatial and motion compensation may be referred toherein as “prediction operations”.

The motion compensator 62 locates a reference block within the framebuffer 58 specified for a particular inter-coded macroblock or codingunit. It does so based on the reference frame information and motionvector specified for the inter-coded macroblock or coding unit. It thensupplies the reference block pixel data for combination with theresidual data to arrive at the recreated video data for that macroblockor coding unit.

A deblocking process may then be applied to a reconstructedframe/slice/picture, as indicated by the deblocking processor 60. Afterdeblocking, the frame/slice/picture is output as the decoded videoframe/picture 16, for example for display on a display device. It willbe understood that the video playback machine, such as a computer,set-top box, DVD or Blu-Ray player, and/or mobile handheld device, maybuffer decoded frames in a memory prior to display on an output device.

Entropy coding is a fundamental part of all lossless and lossycompression schemes, including the video compression described above.The purpose of entropy coding is to represent a presumably decorrelatedsignal, often modeled by an independent, but not identically distributedprocess, as a sequence of bits. The technique used to achieve this mustnot depend on how the decorrelated signal was generated, but may relyupon relevant probability estimations for each upcoming symbol.

There are two common approaches for entropy coding used in practice: thefirst one is variable-length coding, which identifies input symbols orinput sequences by codewords, and the second one is range (orarithmetic) coding, which encapsulates a sequence of subintervals of the[0, 1) interval, to arrive at a single interval, from which the originalsequence can be reconstructed using the probability distributions thatdefined those intervals. Typically, range coding methods tend to offerbetter compression, while VLC methods have the potential to be faster.In either case, the symbols of the input sequence are from a finitealphabet.

A special case of entropy coding is when the input alphabet isrestricted to binary symbols. Here VLC schemes must group input symbolstogether to have any potential for compression, but since theprobability distribution can change after each bit, efficient codeconstruction is difficult. Accordingly, range encoding is considered tohave greater compression due to its greater flexibility, but practicalapplications are hindered by the higher computational requirements ofarithmetic codes.

A common challenge for both of these encoding approaches is that theyare serial in nature. In some important practical applications, such ashigh-quality video decoding, the entropy decoder has to reach very highoutput speed, which can pose a problem for devices with limitedprocessing power or speed.

One of the techniques used together with some entropy coding schemes,such as CAVLC and CABAC, both of which are used in H.264/AVC, is contextmodeling. With context modeling, each bit of the input sequence has acontext, where the context may be given by a certain subset of the otherbits, such as the bits that preceded it, or side information, or both.In a first-order context model, the context may depend entirely upon theprevious bit (symbol). In many cases, the context models may beadaptive, such that the probabilities associated with symbols for agiven context may change as further bits of the sequence are processed.In yet other cases, the context of a given bit may depend upon itsposition in the sequence, e.g. the position or ordinal of a coefficientin a matrix or block of coefficients.

Reference is made to FIG. 3, which shows a block diagram of an exampleencoding process 100. The encoding process 100 includes a contextmodeling component 104 and an entropy coder 106. The context modelingcomponent 104 receives the input sequence x 102, which in this exampleis a bit sequence (b₀, b₁, . . . , b_(n)). In this example illustration,the context modeling component 104 determines a context for each bitb_(i), perhaps based on one or more previous bits in the sequence orbased on side information, and determines, based on the context, aprobability p_(i) associated with that bit b_(z), where the probabilityis the probability that the bit will be the Least Probable Symbol (LPS).The LPS may be “0” or “1” in a binary embodiment, depending on theconvention or application. The determination of the probability mayitself depend upon the previous bits/symbols for that same context.

The context modeling component outputs the input sequence, i.e. the bits(b₀, b₁, . . . , b_(n)) along with their respective probabilities (p₀,p₁, . . . , p_(n)). The probabilities are an estimated probabilitydetermined by the context model. This data is then input to the entropycoder 106, which encodes the input sequence using the probabilityinformation. For example, the entropy coder 106 may be a binaryarithmetic coder. The entropy coder 106 outputs a bitstream 108 ofencoded data.

It will be appreciated each bit of the input sequence is processedserially to update the context model, and the serial bits andprobability information are supplied to the entropy coder 106, whichthen serially entropy codes the bits to create the bitstream 108. Thoseordinarily skilled in the art will appreciate that, in some embodiments,explicit probability information may not be passed from the contextmodeling component 104 to the entropy coder 106; rather, in someinstances, for each bit the context modeling component 104 may send theentropy coder 106 an index or other indicator that reflects theprobability estimation made by the context modeling component 104 basedon the context model and the current context of the input sequence 102.The index or other indicator is indicative of the probability estimateassociated with its corresponding bit.

In some embodiments, the entropy coder 106 may have a parallelprocessing architecture for encoding the input sequence 102. In such anembodiment, the entropy coder 106 may include a plurality of entropycoders each processing a portion of the input sequence 102. In somecases, the input sequence may be demultiplexed an allocated amongst theparallel entropy coders on the basis of the estimated probabilitiesassociated with the respective bits. In other words, a bit from theinput sequence 102 is allocated to one of the parallel entropy coders onthe basis of its estimated probability.

At the decoder, the encoded bitstream is decoded using a reverseprocess. In particular, the decoder performs the same context modelingand probability estimation process to determine the context of the nextreconstructed symbol of the reconstructed sequence. On the basis of thecontext determined for the next reconstructed symbol, an estimatedprobability is determined. The encoded bitstream, which may be made upof codewords output by the entropy coder(s), is decoded to obtaindecoded symbols. The determination of context/probability is interleavedwith the decoding of codewords to obtain decoded symbols correspondingto those estimate probabilities.

In a parallel encoding embodiment, the decoder may be configured todemultiplex the encoded bitstream into a plurality of decodedsubseqeunces, each associated with an estimated probability. The contextmodeling and probability estimation then results in selection of areconstructed symbol from the associated decode subsequence. It will beappreciated that in such an implementation the decoding of the encodedbitstream may be considered de-interleaved from the context modeling andprobability estimation.

It will be appreciated from the detailed description below that thepresent application is applicable to either serial or parallel entropycoding and decoding.

A Context-Based Coding Order

The present application proposes a revised context-based coding order.In many situations, symbols of an input sequence are encoded in theorder in which they appear in the input sequence. The presentapplication proposes a context-based re-ordering of the symbols of theinput sequence. Advantages in coding and decoding throughput may beavailable as a result.

The examples below may refer specifically to video encoding and, inparticular, to the encoding of the sequences sig[i, j] and last[i, j],such as those defined in CABAC as specified in the ITU-T H.264/AVCstandard. The examples illustrate the reordering of the coding/scanningorder to facilitate decoding of more than one symbol at a time. It willbe appreciated that the present application is not limited to theencoding and decoding of these two specific sequences within CABAC; noris it limited to video encoding and decoding. The present applicationdescribes encoding and decoding methods and processes that may beapplied to other data sequences, including video, image, and audio. Themethods and processes herein are applicable to encoding and decodingprocesses that involve a context model which specifies context basedupon information other than the other symbols in the sequence. Theexamples below may refer to a binary source as an example, although thepresent application is more generally applicable to any finite alphabetsource.

In some cases the context model may specify the context of a bit basedupon the bit's position in the sequence. For example, in a block-basedcoefficient encoding scheme such as that specified in ITU-T H.264/AVC,the sequences sig[i, j] and last[i, j] have a context based upon theposition j of the coefficient in the sequence. In another example, thecontext of a bit may depend upon a bit value in a different sequence. Itwill be understood that the CABAC context model is only an exampleillustration and that the present application is not specific toposition-based context models.

The H.264 context model for CABAC encodes DCT coefficients by firstencoding a significance map. The significance map includes twosequences: sig[i, j] and last[i, j]. The first sequence sig[i, j] is abinary sequence indicating whether there is a non-zero coefficient ineach position in the DCT block. The second sequence last[i, j] is abinary sequence mapped to the non-zero coefficients of the DCT block andindicating whether that non-zero coefficient is the last non-zerocoefficient of the DCT block (in the zig-zag scanning order used inH.264).

The H.264 standard specifies a zig-zag scanning order, for encoding thecoefficients of a DCT block. Referring to a 4×4 DCT block for example,the H.264 standard encodes the 16 coefficients in the zig-zag orderillustrated in FIG. 6. As shown in FIG. 12, the scanning order begins inthe upper left corner of the block and follows a zig-zag pattern to thelower right corner. Once arranged in this coding order, the resultingsequence of coefficients for the i^(th) block is X(i, 0), . . . , X(i,15).

A group of n blocks can be represented as a matrix of coefficients asfollows:

X[0, 0] X[0, 1] . . . X[0, 15] X[1, 0] X[1, 1] . . . X[1, 15] . . .X[n-1, 0] X[n-1, 1] . . . X[n-1, 15]

In this matrix X[i, j] denotes the j^(th) coefficient of the i^(th)block.

The H.264/AVC standard uses a scheme in which the blocks are encoded insequence. In other words, each sequence X(i, 0), . . . , X(i, 15) isencoded for i=0, 1, . . . , n−1, in turn. The context model used inH.264/AVC includes determining the two binary sequences, sig[i, j] andlast[i, j] for each vector X[i, j]. The actual values of thecoefficients are then also encoded.

To illustrate by example, consider the following example coefficientsequences X:

-   -   3, 5, 2, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0 i=0    -   6, 4, 0, 3, 3, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0 i=1    -   4, 2, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 i=2

The sequences sig[i, j] and last[i, j] for these coefficient sequencesare as follows:

-   -   sig [0, j]=1, 1, 1, 0, 1, 1, 0, 1    -   last [i, j]=0, 0, 0, 0, 0, 1    -   sig [1, j]=1, 1, 0, 1, 1, 0, 1    -   last [1, j]=0, 0, 0, 0, 1    -   sig [2, j]=1, 1, 0, 0, 1    -   last [2, j]=0, 0, 1

It will be appreciated that, in this example, the last[i, j] sequenceonly includes values when the sig[i, j] is non-zero, and both sequencesterminate after the last non-zero coefficient. Accordingly, last[i, j]will not necessarily include a bit for every bit j in sig[i, j]. It willbe appreciated that the lengths of these sequences may vary depending onthe coefficient values. Finally, it will be appreciated that until oneknows whether the sig[i, j] sequence contains a non-zero bit, one doesnot know whether there is a corresponding bit in the last[i, j] sequenceor not, meaning the encoding and decoding of these sequences isinterleaved by bit position in this example embodiment.

The context model in H.264/AVC specifies context for the bits of thesesequences on the basis of bit position. In other words, the context ofbit j in the sequence sig[i, j] is j (+an offset, if any). Similarly,the context of bit j in the sequence last[i, j] is j (+an offset, ifany). It will be understood that separate contexts are maintained forthe two sequences, meaning that a sig[i, j] bit in the j^(th) positiondoes not have the same context as a last[i, j] bit in the j^(th)position.

In the H.264/AVC example the probability (sometimes termed “state”,“probability state” or “context state”) for a bit is determined based onits context. Specifically, the bit history for that same contextdetermines the probability (e.g. context state/probability state) thatis selected or assigned to that bit. For example, for a bit in thej^(th) position in a given sequence i, its probability state is selectedfrom amongst the 64 possible probabilities based on the history of bitsin the j^(th) position in previous sequences (i−1, etc.).

As described above, the bits are encoded based upon their probabilities.In some example embodiments, parallel encoding may be used and, in someinstances, the parallel encoding may include an entropy coder specificto each probability. In other example embodiments, serial entropyencoding may be used. In either case, the entropy coder encodes symbolsbased on their associated probabilities.

At the decoder, the same context modeling and probability estimationoccurs in reconstructing the sequence. The probability estimation isused to decode the encoded bitstream to obtain reconstructed symbols.Symbols are selected from the decoded portions of the encoded bitstreamand interleaved to form the reconstructed sequence of symbols on thebasis of their associated probabilities and the context model.

It will be understood that a potential bottleneck occurs at the contextmodeling stage since context modeling and probability estimation isperformed symbol-by-symbol. In the specific example of the significancemap for H.264, the coding order described above means that decoding ofsig[i, 0] . . . sig[1, 15] and last[i, 0] . . . last[i, 15] isinterleaved. That is, the value of sig[i, j] impacts whether there is abit for last[i, j] and the value of last[i, j] impacts whether there isa value for sig[i, j+1]. Therefore, the decoder is forced into assessingcontext and determining probability on a bit-by-bit serial basis.

If the decoder were to attempt to evaluate multiple bits at one time,large look-up tables would be required to account for all possibleoutput strings. For example to decode 3 bits together starting at sig[i,j], the following information is needed:

-   -   The state for ctx_sig_offset+j (out of 64 possible values)    -   The state for ctx_last_offset+j+1 (needed if sig[i, j]==1)    -   The state for ctx_sig_offset+j+1    -   The state for ctx_last_offset+j+1 (needed if sig[i, j+1]==1)    -   The state for ctx_sig_offset+j+2 (needed if sig[i, j]==0 and        sig[, j+1]==0).    -   wherein the context of the j^(th) bit in sig is given by        ctx_sig_offset+j, and the context of the j^(th) bit in last is        given by ctx_last_offset+j.

The above information results in 2²¹ possible combinations, which isalmost infeasible to evaluate in practical applications.

Accordingly, the present application proposes a revised context-basedcoding order. In the specific context of video coding, the presentapplication proposes a revised coding order different from the codingorder specified in ITU-T H.264/AVC to facilitate more practical decodingof multiple bits at once.

In accordance with one aspect of the present application, the codingorder may be revised so as to group values by context. In other words,for a group of blocks or for an input sequence, in which at least twosymbols share a common context, the coding order specifies that thosesymbols sharing a common context be processed in sequence. This allowsthe decoder to decode values (e.g. determine probabilities/state andreconstruct the sequence of bits) for multiple bits from the samecontext at once, thereby reducing the number of combinations andpermutations that need to be evaluated at once.

Using the specific example of video encoding outlined above, and theexample matrix of sequences, X[0,j] to X[n−1, j], the coding order maybe “vertical” (e.g., by bit position) instead of horizontal. Forexample, all bits in the j=0 position may be encoded, followed by allbits in the j=1 position, and so on, since all bits in the same bitposition share a common context in this example. By grouping the codingof bits by context, the bottleneck due to interleaving is partiallyalleviated. In other words, in this example through coding by bitposition, the interleaving is made more coarse, since the interleavingis from bit position to bit position, not vertically across sequences.This modification has the potential to improve throughput at the decoderby permitting the decoding of multiple bits at a time.

The following pseudocode illustrates a possible implementation of arevised coding order for encoding the significance map in a videoencoding process:

For j=0, ..., 15   For i=0, ..., n−1     If (last[i, 0] + ... + last[i,j−1] == 0)       Encode sig[i, j] with context_sig_offset + j;     End  End   For i=0, ..., n−1     If (sig[i, j] == 1)       Encode last[i,j] with context_last_offset + j;     End   End End

The sequences X(i, 0), . . . , X(i, 15) may still be formed using thezig-zag scanning order specified by in H.264/AVC, or any other scanningorder specified by the applicable standard. However, a group of nsequences, i=0 to n−1, or blocks are encoded together at the contextmodeling stage by encoding the sequences by bit position j.

At the decoder, this permits the decoder to decode more than one bit ata time using practically-sized lookup tables (for example) where those 4bits share a common context (i.e. bit position). Assuming the 64states/probabilities used in H.264/AVC, a throughput of 4 bits per clockmay be achievable since they would result in possible 1024 combinations.6 bits would produce about 4 k combinations, 8 bits 16 k combinations,and so on. The lookup tables for evaluating the combinations may beencoded in hardware in some implementations.

In some video, image or audio encoding cases fewer probabilities/statesmay be used, which will ease the number of possible combinations andallows for more modestly-sized tables or even higher throughput.

To illustrate by way of an example, consider the example significancemap detailed earlier:

-   -   sig [0, j]=1, 1, 1, 0, 1, 1, 0, 1    -   last [0, j]=0, 0, 0, . . . , 0, 0, . . . , 1    -   sig [1, j]=1, 1, 0, 1, 1, 0, 1    -   last [1, j]=0, 0, . . . , 0, 0, . . . , 1    -   sig [2, j]=1, 1, 0, 0, 1    -   last [2, j]=0, 0, . . . , . . . , 1

The encoding in this example is done in the order sig[0,0], sig[1,0],sig[2,0], last[0,0], last[1,0], last[2,0], sig[1,0], sig[2,0], . . . ,and so on.

At the decoder, the decoder would decode the sequences by commoncontext. First, the decoder decodes the three bits sig[0,0], sig[1,0],and sig[2,0] as a three-bit sequence since they share a common context(j=0). From this operation, the decoder reconstructs the sequence 1, 1,1. Thus, it knows there are non-zero values in each of these positions,which means that last[i, 0] will have a value in each of thesepositions. Accordingly, the decoder then decodes the three bitslast[0,0], last[1,0], and last[2,0] together since they share a commoncontext. From this the decoder reconstructs the sequence 0, 0, 0.

Once the decoder reaches j=2, it decodes sig[i, 2] as 1, 0, 0. From thisoperation, it know that only last[0, 2] will have a value (no valueshave been encoded for last[1,2] or last[2,2]), so it will next decodethe bit for last[0,2].

At j=5, the decoder decodes last[i, 5] as 0, 0, 1. Thus, it knows thatthe sequence sig[2, j] and last[2, j] are terminated. Accordingly, thedecoder next decodes the bits sig[0,6] and sig[1,6] since they share acommon context.

It will be appreciated, that the number of bits that may be processedtogether at the decoder may dwindle as the sequences terminate; however,the revised coding order permits the joint evaluation of a number ofbits. It will also be appreciated that the number of sequences need notbe three as in this example but may include more or fewer sequences. Thelimitation on the number of sequences is the practicality of evaluatingthe probability or state of each of the bits jointly, which may dependon the number of possible probabilities/states in the given contextmodel, and any practical limitations on memory or processor speed interms of evaluating the number of possible combinations.

In the context of video decoding, particularly video decoding based uponthe H.264/AVC context model, within a loop of i, it is possible todecode multiple bits together by following the finite state machine usedin the H.264/AVC context model to estimate probabilities.

In one alternative possible implementation, some compression efficiencyis sacrificed by using a single probability estimation for a group ofbits. At the encoder and decoder, a single probability estimation(probability state) is made for a group of bits having the same contextand that probability is used to encode that group of bits. For example,the group of 3 bits for sig[0, j], j=0, 1, 2, are assigned a probabilityestimate and are all encoded using that same probability estimate. Thisreduces compression efficiency to some degree, but allows the encodingand decoding to avoid the need for tables for evaluating the probabilitycombinations for all permutations of a group of bits.

In yet another possible implementation, the presently describedreordered coding order that groups bits by context may be combined witha reordered context modeling, such as that described in U.S. applicationSer. No. 61/325,806 filed Apr. 19, 2010, the contents of which areincorporated herein by reference. For example, the context modeling forencoding the significance map may be done in parallel with the contextmodeling for encoding motion vector information since they areunrelated; within the context modeling of the significance map thepresent modified coding order may be used to permit the decoding ofmultiple bits at a time. The parallel context modeling may result in anoutput bitstream in which even and odd bits are used for the twoparallel encodings, for example.

It will be understood that the last sequence and sig sequence need notbe interleaved. In some embodiments, the last sequence may contain abin/bit for each bit position up to the position of the last significantcoefficient. This enables independent coding of the last sequence. Adecoder may then decode the last sequence, thereby determining thelocation of the last significant coefficient, and then decode thecorresponding sig sequence. The independence of the last sequence alsomakes it possible to encode and decode a block or series of lastsequences followed by their corresponding block or series of sigsequences. Even in these situations, the present application may beapplied so as to group bits/symbols by context for encoding anddecoding. In another variation, the last sequence may be replaced by abinarized two-dimensional location of the last significant coefficient.

Reference is now made to FIG. 7, which shows a simplified example method800 of entropy encoding an input sequence of bits in accordance with anaspect of the present application. The method 800 includes receiving aninput sequence of bits in operation 802. The input sequence of bits maybe received through reading the input sequence from memory or fromanother computer-readable medium. The input sequence of bits may bereceived from another encoding process, such as, for example, a video,image or audio processing element, such as a spectral transformcomponent for generating transform domain coefficients or quantizedtransform domain coefficients. Other sources of the input sequence ofbits are also contemplated. The input sequence of bits may, in someembodiments, include a plurality of transform blocks or samples.

In operation 804, the method 800 includes determining a context for eachof the bits of the input sequence. The context is prescribed by acontext model, which specifies context for a bit based on informationother than the other bits in the input sequence. In one exampleembodiment, the context may be based upon bit position within a block,set or sample of bits in the input sequence.

In operation 806, the bits may be grouped by their respective contexts.For each context, the method 800 includes determining the probability ofeach bit having that context and encoding those bits in accordance withtheir probabilities. The determination of probabilities is done on acontext-by-context basis, meaning that the bits of the input sequencehaving the same context are processed consecutively. Estimating theprobability of a given bit may include selecting from a set ofpredetermined candidate probabilities based upon the bit history for thecontext associated with that bit. Bits are encoded in accordance withtheir estimated probability by an entropy coder. In one exampleembodiment, more than one entropy coder is provided in parallel, andbits are allocated between the entropy coders on the basis of theirestimated probabilities for encoding. In one example embodiment, anentropy coder is provided for each probability.

The encoded bitstream that results from the encoding of the bits isoutput in operation 808.

Reference is now made to FIG. 8, which shows, in flowchart form, anexample method 850 of decoding a bitstream of encoded data. The method850 includes a first operation 852 of receiving the encoded data.Although the example herein refers to “bits”, it will be appreciatedthat it is more generally applicable to “symbols” of a finite alphabet.

The received bitstream of encoded data is decoded in operation 854 toobtain decoded subsequences of bits each associated with a probability.In a serial implementation, it will be appreciated that operation 854 isinterleaved with the determination of context and estimation ofprobability described below. In a parallel implementation operation 854might be performed separately from the determination of context andestimation of probability to provide the decoded subsequences of bits.

The method 850 includes an operation 856 of determining the context oftwo or more bits to be reconstructed. In particular, operation 856involves identifying a group of consecutively encoded bits based uponthe group of bits having the same context. In other words, the contextis determined and, based upon the coding order and bit history, if any,the decoder knows how many bits having the same context would have beenconsecutively encoded. For example, in the case of video encoding asillustrated above, when decoding j=0 the decoder may expect n bits areencoded for sig[i, j] since i will range from 0 to n−1. Later bitpositions may have fewer bits with the same context, as may bedetermined by how many sequences have terminated and/or whether thereare non-zero bits in the sig vector. The decoder may be configured toimpose a maximum number of bits to assess in a group. When the number ofbits having a given context exceeds the maximum that the decoder isconfigured to handle at one time, the decoder may process a givencontext in two or more groups.

In operation 858, the method includes estimating the probabilities ofeach bit in the group of consecutive bits having the same context. Theestimation of the bit probabilities may include use of a look-up table,decision tree, state machine, or other mechanisms for efficientdetermination of probabilities of more than one bit at the same time.

In the serial case, the group of bits may then be decoded from theencoded data on the basis of their estimated probabilities. Afterdecoding a series of groups of bits each corresponding to one of thecontexts, the decoder reorders the bits as described below in connectionwith operation 862.

Returning to the parallel decoding example, operation 860 involvesreconstructing the group of sequentially encoded bits by selecting bitsfrom the decoded subsequences of bits based on the estimatedprobabilities determined in operation 858.

In operation 862 the reconstructed group of bits may be rearranged basedupon the coding order.

It will be appreciated that operations 854, 856, 858 and 860 may berepeated in a cycle to obtain a plurality of groups of bits for aplurality of contexts and the rearrangement in operation 862 may includeinterleaving bits or allocating them so as to reconstruct the inputsequence of bits in light of the context-based coding order used at theencoder.

Reference will now be made to FIG. 9, which shows a more detailedflowchart illustrating an example method 820 of encoding quantizedtransform domain coefficients in a video encoding process. The inputsequence of quantized transform domain coefficients is received inoperation 822 of the method 820. The input sequence includes a set orgroup of n blocks or sequences of quantized transform domaincoefficients.

In operation 824, the context model, such as, for example CABAC inH.264/AVC, is applied to generate a significance map for each of the nblocks or sequences. The significance map for each block includes thesequences sig[i, j] and last[i, j]. In operation 826, the indexes i andj are initialized to zero. Index i is an index of the sequence/block,wherein i ranges from 0 to n−1. Index j is an index of the bit position,wherein in this example j ranges from 0 to 15 on the basis that thepresent example applies to 4×4 transform blocks. Other sizes may be usedin other applications.

In a loop of operations 828, 830 and 831, the probability of sig[i, j]is estimated and the bit is encoded accordingly, which may includeallocating the bit to the entropy encoding associated with theprobability estimate. The loop of operations 828, 830 and 831 cyclesthrough each sig bit in all blocks/sequences for a given bit position,i.e. until i=n−1. Although not explicitly shown, for a given position joperation 828 is conditional on the sequence/block sig i not havingalready terminated.

The method 820 then includes a loop of operations 834, 836, 840, and842. The loop estimates the probability of and encodes the last[i, j]bits for a bit position j. It will be noted that the last[i, j] bit isonly encoded if the corresponding sig[i, j] bit is equal to 1.

After operation 840 reaches the last of the sequences/blocks, i.e.i=n−1, the method 820 evaluates whether it has reached the last bitposition, i.e. whether j=15, and if not then the method 820 incrementsbit position index j in operation 846, resets block index i, and returnsto operation 828. The method 820 results in the encoding of the sig[i,j] and last[i, j] sequences, i.e. the significance maps, for the groupsof n blocks/sequences in an encoding order based on grouping bitstogether sequentially for encoding that share the same context, whereinencoding means determining the probability of each bit and entropyencoding it accordingly.

In a variation to method 820, the group of last[i, j] bits for bitposition j may be encoded for i=0 to n, for j=0 until the sequencesterminate. The sig sequences are then encoded (except the last bit,which is known to be a 1 and is at a known location based on the lastsequences) using the context-based coding order.

Reference is now made to FIG. 10, which shows, in flowchart form, amethod 870 for decoding a bitstream of encoded video data. The encodedvideo data includes codewords generated by one or more entropy encoders.Each codeword may be decoded to obtain a decoded subsequence of bitsassociated with an estimated probability. In this example, theprobabilities are selected from a predetermined set of candidateprobabilities such as, for example, the 64 probabilities or statesdefined in CABAC for H.264/AVC.

The method 870 includes receiving the bitstream of encoded video data inoperation 872. The encoded video data may be received by way of abroadcast transmission or download over a network, or by reading thebitstream of encoded video data from a computer-readable medium, such asa compact disc or other optical or magnetic storage medium. The encodedvideo data is decoded in operation 874 by decoding the codewords togenerate the subsequences of bits each associated with a probability. Itwill be appreciated that operation 874 may be interleaved with theoperations described below such that the decoding occurs as probabilityestimations are made and further decoded bits associated with thatprobability are required for the reconstruction.

In operation 876 the indexes i and j are initialized to zero. Theindexes have the same meaning explained above in connection with FIG.15.

In operation 878, for a given bit position j, i.e. for a common context,the probability is estimated for each bit sig[i, j] in theblocks/sequences i=0 to n−1. For a given block or sequence i, this isconditional on the sequence/block not having already terminated, whichcan be determined by whether a last[i, <j] bit equal to 1 has beenidentified for earlier bit positions such as j−1, j−2, etc. A look-uptable, decision tree, or other data structure or mechanism may be usedto nearly simultaneously estimate the probabilities of two or more bitsat a time.

It will be appreciated that in some embodiments, the decoder may not beconfigured to evaluate at the same time all n bits that may have thesame context. In this case, the decoder may evaluate the n bits in twoor more groups of multiple bits.

In operation 880, having estimated the probabilities of the bits sig[i,j] across multiple sequences i for a given j, bits are selected from thedecode subsequences of bits on the basis of those estimatedprobabilities to form or reconstruct those bits of sig[i, j].

Similar operations 882 and 884 are then performed to reconstruct bits oflast[i, j] for a given bit position across a group of sequences/blocksi.

These operations 878, 880, 882, 884 are repeated as j is incremented inoperation 888 until j reaches its limit as indicated by operation 886.It will be appreciate that this loop may be ended earlier if all sigsequences have terminated at earlier bit positions. The vectors sig[i,j] and last[i, j] that form the significance maps for the group ofsequences/blocks are thus reconstructed.

In a variation of method 870, the decoder first decodes all n of thelast sequences, which are in the bitstream in a context-based order asdescribed above. The decoder then decodes all n of the sig sequences inthe context-based order.

It will be appreciated that the foregoing example methods areillustrations of a specific example application for encoding anddecoding significance maps, such as those prescribed in H.264. Thepresent application is not limited to that specific example application.

Reference now made to FIG. 4, which shows a simplified block diagram ofan example embodiment of an encoder 900. The encoder 900 includes aprocessor 902, memory 904, and an encoding application 906. The encodingapplication 906 may include a computer program or application stored inmemory 904 and containing instructions for configuring the processor 902to perform steps or operations such as those described herein. Forexample, the encoding application 906 may encode and output bitstreamsencoded in accordance with the modified context-based coding orderprocess described herein. The encoding application 906 may include anentropy encoder 26 configured to entropy encode input sequences andoutput a bitstream using one or more of the processes described herein.It will be understood that the encoding application 906 may be stored inon a computer readable medium, such as a compact disc, flash memorydevice, random access memory, hard drive, etc.

In some embodiments, the processor 902 in the encoder 900 may be asingle processing unit configured to implement the instructions of theencoding application 906. It will further be appreciated that in someinstances, some or all operations of the encoding application 906 andone or more processing units may be implemented by way ofapplication-specific integrated circuit (ASIC), etc.

Reference is now also made to FIG. 5, which shows a simplified blockdiagram of an example embodiment of a decoder 1000. The decoder 1000includes a processor 1002, a memory 1004, and a decoding application1006. The decoding application 1006 may include a computer program orapplication stored in memory 1004 and containing instructions forconfiguring the processor 1002 to perform steps or operations such asthose described herein. The decoding application 1006 may include anentropy decoder 1008 configured to receive a bitstream encoded inaccordance with the modified context-based coding order processdescribed herein, and to extract encoded subsequences from the bitstreamand perform multi-bit probability estimation for decoding of groups ofbits having the same context, as described herein. It will be understoodthat the decoding application 1006 may be stored in on a computerreadable medium, such as a compact disc, flash memory device, randomaccess memory, hard drive, etc.

In some embodiments, the processor 1002 in the decoder 1000 may be asingle processing unit configured to implement the instructions of thedecoding application 1006. In some other embodiments, the processor 1002may include more than one processing unit capable of executinginstructions in parallel. The multiple processing units may be logicallyor physically separate processing units. It will further be appreciatedthat in some instances, some or all operations of the decodingapplication 1006 and one or more processing units may be implemented byway of application-specific integrated circuit (ASIC), etc.

It will be appreciated that the decoder and/or encoder according to thepresent application may be implemented in a number of computing devices,including, without limitation, servers, suitably programmed generalpurpose computers, set-top television boxes, television broadcastequipment, and mobile devices. The decoder or encoder may be implementedby way of software containing instructions for configuring a processorto carry out the functions described herein. The software instructionsmay be stored on any suitable computer-readable memory, including CDs,RAM, ROM, Flash memory, etc.

It will be understood that the encoder and decoder described herein andthe module, routine, process, thread, or other software componentimplementing the described method/process for configuring the encodermay be realized using standard computer programming techniques andlanguages. The present application is not limited to particularprocessors, computer languages, computer programming conventions, datastructures, other such implementation details. Those skilled in the artwill recognize that the described processes may be implemented as a partof computer-executable code stored in volatile or non-volatile memory,as part of an application-specific integrated chip (ASIC), etc.

Certain adaptations and modifications of the described embodiments canbe made. Therefore, the above discussed embodiments are considered to beillustrative and not restrictive.

What is claimed is:
 1. A method for decoding encoded data to reconstructa sequence of symbols, wherein the encoded data has been encoded inaccordance with a context model, wherein the context model specifies acontext for each symbol, the method comprising: for each of two or moreof the contexts, decoding a portion of the encoded data to obtain agroup of consecutive symbols having that context, wherein decodingincludes jointly determining in parallel, for the consecutive symbols inone of the groups, a probability associated with each symbol, anddecoding each of the consecutive symbols from the encoded data based onits determined probability; and reconstructing the sequence of symbolsby reordering the symbols of the groups of consecutive symbols, whereinreordering the symbols results in a series of sequences, and wherein thecontext for each symbol is associated, at least in part, with thatsymbol's position in its sequence.
 2. The method claimed in claim 1,wherein the reordering is based upon a coding order specified by thecontext model.
 3. The method claimed in claim 2, wherein each of thegroups of consecutive symbols is associated with one of the contextsdefined in the context model, and wherein the reordering includesinterleaving the symbols of the groups to undo the context-basedgrouping.
 4. The method claimed in claim 1, wherein the group ofconsecutive symbols contains symbols having the same symbol position intheir respective sequences.
 5. The method claimed in claim 1, whereinthe encoded data comprises a plurality of significance maps, each mapincluding a significant-coefficient sequence and a last-coefficientsequence, and wherein each group of consecutive symbols comprises bitshaving the same bit position in a series of the significant-coefficientsequences or the last-coefficient sequences.
 6. A decoder for decodingencoded data to reconstruct a sequence of symbols, wherein the encodeddata has been encoded in accordance with a context model, wherein thecontext model specifies a context for each symbol, the decodercomprising: a processor; a memory; and a decoding application stored inmemory and containing instructions for configuring the processor todecode the encoded data by for each of two or more of the contexts,decoding a portion of the encoded data to obtain a group of consecutivesymbols having that context, wherein decoding includes jointlydetermining in parallel, for the consecutive symbols in one of thegroups, a probability associated with each symbol, and decoding each ofthe consecutive symbols from the encoded data based on its determinedprobability, and reconstructing the sequence of symbols by reorderingthe symbols of the groups of consecutive symbols, wherein reordering thesymbols results in a series of sequences, and wherein the context foreach symbol is associated, at least in part, with that symbol's positionin its sequence.
 7. The decoder claimed in claim 6, wherein theprocessor is configured to reorder the symbols based upon a coding orderspecified by the context model.
 8. The decoder claimed in claim 7,wherein each of the groups of consecutive symbols is associated with oneof the contexts defined in the context model, and wherein the processoris configured to reorder by interleaving the symbols of the groups toundo the context-based grouping.
 9. The decoder claimed in claim 6,wherein the group of consecutive symbols contains symbols having thesame symbol position in their respective sequences.
 10. The decoderclaimed in claim 6, wherein the encoded data comprises a plurality ofsignificance maps, each map including a significant-coefficient sequenceand a last-coefficient sequence, and wherein each group of consecutivesymbols comprises bits having the same bit position in a series of thesignificant-coefficient sequences or the last-coefficient sequences. 11.A method of entropy encoding an input sequence of symbols using acontext model, wherein the context model specifies a context for eachsymbol, the method comprising: reordering the input sequence to groupthe symbols on the basis of context to form groups of symbols, eachgroup being associated with a respective one of the contexts; for eachof the groups of symbols, jointly determining in parallel, for thesymbols in the group, a probability associated with each symbol; and foreach of the groups of symbols, encoding consecutively the symbols inthat group based upon their determined probabilities, to produce encodeddata, wherein the input sequence includes a series of sequences andwherein the context for each symbol is based, at least in part, uponthat symbol's position in its sequence.
 12. The method claimed in claim11, wherein the group of consecutive symbols contains symbols having thesame symbol position in their respective sequences.
 13. The methodclaimed in claim 11, wherein the input sequence comprises a plurality ofsignificance maps, each map including a significant-coefficient sequenceand a last-coefficient sequence, and wherein each group of symbolscomprises bits having the same bit position in a series of thesignificant-coefficient sequences or the last-coefficient sequences. 14.An encoder for entropy encoding an input sequence of symbols using acontext model, wherein the context model specifies a context for eachsymbol, the encoder comprising: a processor; a memory; and an encodingapplication stored in memory and containing instructions for configuringthe processor to encode the input sequence by: reordering the inputsequence to group the symbols on the basis of context to form groups ofsymbols, each group being associated with a respective one of thecontexts, for each of the groups of symbols, jointly determining inparallel, for the symbols in the group, a probability associated witheach symbol, and for each of the groups of symbols, encodingconsecutively the symbols in that group based upon their determinedprobabilities, to produce encoded data, wherein the input sequenceincludes a series of sequences and wherein the context for each symbolis based, at least in part, upon that symbol's position in its sequence.15. The encoder claimed in claim 14, wherein the group of consecutivesymbols contains symbols having the same symbol position in theirrespective sequences.
 16. The encoder claimed in claim 14, wherein theinput sequence comprises a plurality of significance maps, each mapincluding a significant-coefficient sequence and a last-coefficientsequence, and wherein each group of symbols comprises bits having thesame bit position in a series of the significant-coefficient sequencesor the last-coefficient sequences.